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ABSTRACT 
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2001 school year. The final evaluation focuses primarily on the Reading First 
program’s effect on students’ reading achievement. Information comparing the 
Reading First program to other reading initiatives is also provided. The 
achievement data confirm that the program has its greatest effect on first 
grade students and low achievers. The data also showed that Reading First and 
non Reading First teachers shared similar philosophies toward teaching 
reading and employed similar instructional practices in the classroom. In 
terms of achievement, scores on the Iowa Tests of Basic Skills (ITBS), 
Stanford Achievement Test Level 9 (SAT-9), and the Criterion-Referenced 
Competency Scores (CRCT) all improved. Recommendations center on: funding; 
student assessment; better continuity across the curriculum; staff 
development; and parent involvement. The following items are appended: (1) 

Program Guidelines; (2) Survey Instruments and Discussion Group Protocols; 
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Executive Summary 



As part of the final year evaluation of the Reading First program, the Georgia Department of 
Education (GDOE) continued to follow the achievement of the original group of Reading First 
schools (those that implemented the program during the 1998-99 school year) during the 2000- 
2001 school year. The final evaluation focuses primarily on the Reading First program’s effect 
on students’ reading achievement. Information comparing the Reading First program to other 
reading initiatives is also provided. 

The results from the final evaluation provide additional support for the year one and year two 
evaluation findings. The achievement data confirm that the Reading First program has its 
greatest effect on first grade students and low achievers. 

The data also showed that Reading First and non Reading First teachers shared similar 
philosophies toward teaching reading and employed similar instructional practices in the 
classroom. Although teachers and administrators at both Reading First and non Reading First 
schools felt that parental involvement was low, Reading First and non Reading First parents 
indicated that they engaged in similar reading related activities with their children and 
participated in school activities at approximately the same rate. 

In terms of achievement, the major findings are listed below. 

Iowa Tests of Basic Skills (ITBS) Results 

• The average second grade reading vocabulary and comprehension ITBS scores 
consistently improved from 1998 to 2001. 

• The percentage of Reading First students scoring at or below the 25 th percentile on the 
vocabulary and comprehension sub-tests continually decreased in grades one, two and 
three from 1998 to 2001. 

• The percentage of first grade Reading First students scoring at or above the 75 th 
percentile on the reading vocabulary and comprehension sub-tests continually increased 
from 1998 to 2001. 

• The percentage of second grade students scoring at or above the 75 percentile in 
vocabulary and comprehension increased from 2000 to 200 1 . 

• The percentage of third grade students scoring at or above the 75 percentile in reading 
comprehension continually increased from 1 999 to 200 1 . 

• The percentage of third grade students scoring at or above the 75 th percentile in reading 
vocabulary increased from 2000 to 2001. 

Stanford-9 (SAT-9) Results 

• Reading First students scored higher on the reading vocabulary and comprehension 
portions of the SAT-9 in 2001 when compared to a sample of their non Reading First 
peers. 
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• When compared to a sample of non Reading First students, fewer Reading First students 
scored at or below the 25‘" percentile in both vocabulary and comprehension on the SAT- 
9. 

Criterion-Referenced Competency Scores (CRCT) 

• Fourth grade Reading First students improved their Criterion-Referenced Competency 
Tests (CRCT) reading performance in 2001 when compared to 2000. 

• The percentage of Reading First students not meeting the standard on the CRCT 
decreased by 10-percentage points in 2001. 

• There was a 10-percentage point gain in the number of Reading First students meeting or 
exceeding the standard in 2001 when compared to 2000. 

• Of the 302 Reading First schools with two years of CRCT data, 83 percent showed 
increases in the percentage of students meeting or exceeding the standard. 



As a result of this evaluation, several recommendations for continued improvement of Reading 
First and reading, in general, were made. The recommendations center mainly on funding, 
student assessment, better continuity across the curriculum, staff development, and parent 
involvement. The data from the final evaluation of the Reading First program provides support 
for its continued use, particularly with early elementary and low performing students. 
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Parti: Overview 



In the fall of 1998, the Georgia Department of Education began a statewide evaluation of 
the effectiveness of the Reading First program. The evaluation included all 351 
elementary schools that implemented the program during the 1998 school year. The three 
evaluation questions are presented below: 

• How, specifically, is Reading First being implemented in Georgia? 

• To what extent does Reading First affect student’s academic success? 

• How are the effects of Reading First influenced by the type of reading program 
and materials used by individual schools and the extent/level of implementation 
of the original program? 

The purpose of Georgia’s Reading First program is to increase reading achievement 
among all students in grades K through three by focusing reading instruction on the 
following: 1) quality literature, 2) direct, systematic phonics, 3) high frequency sight 
vocabulary, and 4) reading comprehension strategies in the content areas of science and 
social studies (see Appendix A for a complete copy of the program guidelines). 

The overall purpose of the evaluation was to provide information that program 
stakeholders could use for decision-making regarding program policies and program 
improvement. The evaluation design consisted of a three-year longitudinal study in 
which the academic achievement of Reading First students and the extent to which the 
program was properly implemented at the participating elementary schools was assessed 
each year to determine the program’s effectiveness. It is important to note that while the 
original sample had 351 Reading First schools, during the second year of the program, 24 
schools elected to discontinue the program. During the 2000-01 school year, an 
additional 19 schools discontinued the program, therefore, the statewide evaluation 
includes only those schools that continued the program for the full three years. 

The first evaluation question was the primary focus of the year one evaluation. The most 
important findings from that study were: 

• Reading First students in grades K through three scored higher on the vocabulary 
and comprehension portions of the Iowa Tests of Basic Skills (ITBS) than 
students who attended the same schools the year before the program was 
implemented. 

• The Reading First program has the greatest effect on student achievement for first 
graders and low performing second and third grade students. 

• Many schools were not able to fully implement the program at the beginning of 
the 1998 school year. 
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The second year evaluation focused, for the most part, on addressing the second and third 
evaluation questions. The data from that study yielded results that supported the findings 
of the year one evaluation. Some of the most important findings were: 

• The proportion of Reading First students scoring at or below the 25 th percentile 
continuously decreased over the two-year period in both vocabulary and 
comprehension at all grade levels. 

• The proportion of Reading First students in first grade scoring at or above the 75 th 
percentile in both vocabulary and comprehension increased from 1998 to 2000. 

• The proportion of second grade Reading First students scoring at or above the 75 th 
percentile in vocabulary increased from 1998-2000. 

• The average first, second and third grade ITBS scores at Reading First schools have 
improved in both comprehension and vocabulary from 1998-2000. 

• Although their overall ITBS scores were lower, third grade Reading First students 
showed greater gains in vocabulary and comprehension over the two-year period than 
the state of Georgia as a whole. 

• By the beginning of the 1999-2000 school year, the Reading First program was fully 
implemented. 

The main focus of this final evaluation report is the second evaluation question: “To what 
extent does Reading First affect student’s academic success?” Having sufficiently 
addressed implementation issues during the year one and two evaluations, this report 
examines longitudinal effects of the program, and compares the reading achievement of 
Reading First students with their peers attending non Reading First schools. The extent to 
which the different types of materials used as part of the Reading First program impact 
achievement and vary by implementation was addressed in the year two evaluation 
report. Prescriptive information about the use of Reading First and other reading 
programs throughout the state of Georgia was collected through discussion groups with 
Reading First and non Reading First teachers and administrators, and through the use of 
surveys with the aforementioned groups as well as elementary school parents. This 
information is used in this report to place achievement in the context of program 
implementation. 

Methodology 

The data collected as part of this evaluation came from multiple sources: discussion 
groups with program administrators and teachers of Reading First and non Reading First 
schools; surveys of Reading First and non Reading First teachers and parents; and 
achievement data from a variety of sources. 

The discussion groups were conducted by staff from the Occupational Research Group at 
the University of Georgia. The discussion groups included two groups of program 
stakeholders: teachers and administrators for a total of 180 participants representing 16 
school districts and one Regional Educational Services Agency (RESA). 
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Surveys were administered through the Research, Evaluation and Testing Division of the 
Georgia Department of Education to approximately 7,000 teachers and 3,000 parents at a 
sample of Reading First and non Reading First schools in the spring of 2001. The 
purpose of the surveys was to uncover differences and similarities in instructional 
practices, teaching philosophies, and parental involvement between the two groups. 

The achievement data used in the present study came from multiple sources. Riverside 
Publishing Company provided Iowa Tests of Basic Skills (ITBS) scores for all students 
participating in the Reading First program. Four years worth of ITBS scores were 
available for Reading First students (1998, 1999, 2000, 2001). Harcourt Educational 
Measurement, Inc. provided third grade Stanford-9 (SAT-9) reading scores. These scores 
were examined to compare achievement between a sample of Reading First and non 
Reading First schools. Finally, Criterion-Referenced Competency Tests (CRCT) reading 
scores were used to compare the achievement of fourth grade students who participated 
in the Reading First program for one or two years. 

A triangulation of data collection methods was used to provide GDOE with data from 
multiple sources that are mutually supportive. This technique adds more credibility and 
confidence to the study. A detailed description of discussion groups protocols and the 
survey instruments are provided in Appendix B. 
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Part II: Student Academic Achievement 



The main purpose of this final evaluation report is to examine the extent to which 
participating in the Reading First program has effected students’ reading achievement 
over the last three years. In order to do so, several sources of achievement data are 
analyzed and presented in this section of the report. First, the reading comprehension and 
vocabulary Iowa Tests of Basic Skills (ITBS) scores for Reading First students are 
examined. The ITBS has been administered each spring to all Reading First students as a 
requirement of the program. ITBS data, therefore, are available for a four-year period that 
began in 1998 (the year before the program was implemented) and ended in 2001, and 
will be used to examine the longitudinal effects of the program on student achievement. 
Second, the reading portion of the Criterion-Referenced Competency Tests (CRCT) is 
used to compare the performance of fourth grade students who participated in the 
Reading First program for one or two years. Finally, scores on the reading section of the 
Stanford-9 (SAT-9) for a sample of Reading First and non Reading First schools are 
examined to determine whether there are significant differences in achievement. 

Demographic Characteristics of Reading First Schools 

Demographic information on the schools participating in the Reading First program has 
not been available in time for use in previous evaluations. This information is typically 
obtained from the State Student Information System and includes information about 
eligibility for free and/or reduced lunch (a proxy for socio-economic status), participation 
in other compensatory educational programs (i.e.; Title I, remedial education, special 
education) and ethnicity. Since this is the final evaluation of the Reading First program, it 
is important to describe the demographic characteristics of participating schools and, 
where possible, discuss the implications of specific demographic characteristics on 
achievement. It is important to note, however, that the demographic information 
presented in this report was obtained from the 2000 database and the demographic make- 
up of Reading First schools may have changed during 2001 . Furthermore, individual 
information was not available for Reading First students, therefore, only school-level 
demographic data are presented. 

Chart 1 shows the percentage of Reading First students in each of the three largest ethnic 
groups: White, African American and Hispanic. As shown, nearly one-half of the 
students attending Reading First schools are African American (48%) while the 
remaining majority of students are White (46%). A small percentage (4%) of students 
attending Reading First schools are Hispanic, and the remaining two percent of Reading 
First Schools are made up of students of other ethnic backgrounds. 
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Chart 1. Ethnic Breakdown of Reading First Schools 



Percent 




□ White 

M African American 

□ Hispanic 



Charts 2, 3, and 4 show the percentage of Reading First schools that participate in the 
Title I program, percentage of students eligible for free and/or reduced price lunch, and 
the percentage of students enrolled in compensatory programs such as remedial and 
special education. 
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Chart 3. Percentage of Students Attending Reading First Schools Eligible for Free 
and Reduced Lunch 



Percent 
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Chart 4. Percentage of Students Attending Reading First Schools Participating in 
Compensatory Education Programs 




The data in Chart 2 show that the majority of Reading First schools (86%) have either 
the targeted assistance or school-wide Title I programs in place. In addition, Chart 3 
shows that a large percentage of students attending Reading First schools (64% ) are 
eligible for either free or reduced priced lunch. With free or reduced lunch eligibility 
being the most appropriate indicator of socio-economic status available, these data 
suggest that over one-half of the students attending Reading First schools are from lower 
income families. Finally, Chart 4 shows that 12% of students attending Reading First 
Schools are enrolled in special and remedial education programs. 

Based on the demographic information that is available, the Reading First program 
appears to be implemented in schools with high levels of need. This evaluation can not, 
however, explore the extent to which these demographic factors impact individual student 
achievement. 

Academic Achievement and ITBS 

This section of the report contains several analyses of ITBS data. First, the average 
vocabulary and comprehension national percentile rank (NPR) scores of first, second and 
third graders at Reading First schools are compared for 1998 (pre-implemenation), 1999 
(year one), 2000 (year two) and 2001 (year three). Next, differences in the proportion of 
low and high performing Reading First students over the three year period is assessed by 
comparing the percentage of students in lower and upper quartiles (under the 25 th and 
over the 75 th percentiles). Finally, a longitudinal analysis which examines the 
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performance of Reading First students over the three year period is presented. The school 
level ITBS scores for Reading First schools over the four year period (1998-2001) are 
contained in Appendix C. 

Comparison of Pre and Post ITBS Scores by Grade. The first analysis presented 
answers the question: “Do Reading First students perform better or worse than students 
who attended the same schools in the previous three years?” The purpose of this kind of 
cross-sectional analysis is to determine whether implementation of the Reading First 
program has had a school wide effect. In other words, do first grade students in 2001 
perform better than their same grade peers did in 1998, 1999 and 2000? Although in 
2001, first graders have only participated in the program for one year (beginning in 
kindergarten), they are attending schools where the program has been in place for three 
years. If implementing the Reading First program has improved reading instruction, it is 
likely that the overall performance of students in 2001, regardless of their individual 
length of program participation, would be better when compared to their peers who 
attended the schools before the program was implemented or when the program had been 
in place for a shorter period of time. The average first, second, and third grade 
vocabulary and comprehension percentile scores from 1998 to 2001 are provided in Table 
1. 



Table 1. National Percentile Rank Scores for Reading First Schools by Grade, 
1998-2001 





1998 


1999 


2000 


2001 


1 st 

Grade 

Reading 

Vocabulary 


48 


53 


57 


57 


Reading 

Comprehension 


55 


59 


63 


63 


2 nd Grade 
Reading 
Vocabulary 


44 


46 


48 


50 


Reading 

Comprehension 


50 


52 


54 


56 


3 rd Grade 
Reading 
Vocabulary 


38 


40 


40 


38 


Reading 

Comprehension 


41 


41 


43 


41 



The data show mixed results. In first grade, the average vocabulary and comprehension 
scores did not change from 2000 to 2001. These scores, however, are higher than they 
were in both 1998 and 1999, pre- implementation and year one of the program. It is 
possible that by 2001, the Reading First program was securely in place and the kind of 



Georgia Department of Education 
Linda C. Schrenko, State Superintendent of Schools 
November 2001 • Page 9 of 42 
All Rights Reserved 



improvement that marked the first two years of implementation, stabilized by year three 
for first grade students. On the other hand, second grade Reading First students 
consistently improved their reading achievement from 1998 to 2001 by two percentile 
points in both vocabulary and comprehension. In third grade, where achievement has 
been sporadic over the duration of the program, vocabulary and comprehension scores 
decreased in 2001, and are consistent with third grade performance in 1998, before the 
program was in place. Data from the previous two evaluations have indicated that the 
instruction in third grade needed to become more rigorous for average and high achieving 
third grade students. A comprehension module was designed to provide this rigor and 
was implemented as part of the Reading First program during the 2000-01 school year. It 
is possible that one year of implementation was not long enough to see the effects of 
more rigorous instruction at grade three, therefore, third grade Reading First students did 
not show improved performance. In addition, introducing a new program component 
(i.e., the comprehension module) could by itself cause the drop-off in scores as teachers 
get used to the new instructional practice. 

Achievement Comparisons by Quartile. The previous section assessed differences in 
ITBS performance for each grade level over the three year period at Reading First 
schools. The following analysis examines whether the percentage of low and high 
performing students has changed since the beginning of the Reading First program. The 
purpose of this type of analysis is to determine whether the proportion of low performing 
students had decreased since the inception of the Reading First program. If the program 
is having its intended effect, it could be assumed that the percentage of students with poor 
reading achievement should decrease while the percentage of high performing students 
should increase. In order to establish the proportion of low and high performing students, 
ITBS data were used to determine how many students scored below the 25 th percentile 
(lower quartile) or above the 75 th percentile (upper quartile) in comprehension or 
vocabulary from 1998-2001. Charts 5 and 6 and Table 2 contain these data. 

Chart 5. Percentage of Reading First Students Scoring At or Below the 25 th 
Percentile by Grade, 1998-2001 

□ 1998 B 1999 D2000 0 2001 
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Chart 6. Percentage of Reading First Students Scoring At or Above the 75 th 
Percentile by Grade, 1998-2001 

0 1998 H 1999 D2000 ^ 2001 
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Table 2. Percentage of Reading First Students Scoring At or Below the 25 th 
Percentile and At or Above the 75 th Percentile by Grade, 1998-2001 





1998 


1999 


2000 


2001 


Low Achieving Students 
(At or Below the 25 ,h 










Percentile) 










1 st Grade 










Reading Vocabulary 


32.3 


26.7 


23.9 


21.9 


Reading Comprehension 
2 nd Grade 


25.6 


20.9 


18.0 


11.2 


Reading Vocabulary 


30.4 


28.9 


25.4 


23.9 


Reading Comprehension 
3 rd Grade 


27.3 


25.3 


22.7 


18.5 


Reading Vocabulary 


35.6 


33.2 


32.8 


28.0 


Reading Comprehension 


34.3 


32.9 


32.4 


30.7 


High Achieving Students 
(At or Above the 75 ,h 










Percentile) 










1 st Grade 










Reading Vocabulary 


24.7 


25.5 


28.4 


33.6 


Reading Comprehension 
2 nd Grade 


23.9 


30.1 


33.6 


42.0 


Reading Vocabulary 


18.1 


18.6 


21.7 


25.5 


Reading Comprehension 
3 rd Grade 


20.6 


22.4 


20.2 


25.0 


Reading Vocabulary 


12.8 


13.4 


12.6 


15.8 


Reading Comprehension 


12.7 


12.5 


12.7 


14.8 



The data show a consistent pattern of improvement at all three grade levels in the 
percentage of low performing Reading First students (at or below the 25 th percentile). 
Since 1998, there have been fewer students scoring in the lower quartile in vocabulary 
and comprehension each year. This suggests that the Reading First program is meeting 
its intended goal of providing students with reading skills that improve their reading 
achievement. These data also support the finding that the Reading First program has its 
biggest effect on low performing students. 

A somewhat similar pattern exists when comparing the percentage of students scoring at 
or above the 75 th percentile. In first grade, there has continuously been more students 
scoring at or above the 75 th percentile since 1998. Again, the data support the finding that 
the Reading First program provides the greatest benefit to first grade students. In grade 
two, there has been consistent improvement in vocabulary but mixed comprehension 
results. While there was an increase in the percentage of students scoring at or above the 
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75 th percentile in comprehension from 2000 to 2001, there was a decrease from 1999 to 
2000. This same pattern exists in third grade vocabulary scores. More students scored at 
or above the 75 th percentile from 2000 to 2001, while there was a decrease from 1999 to 
2000, in vocabulary. In comprehension, scores increased consistently from 1999 to 2001, 
however, the data show a slight decrease from 1998 to 1999. 

Overall, the data presented support the findings of the previous two evaluations. Reading 
First appears to have the biggest impact on first grade reading achievement and the 
reading achievement of low performing second and third grade students. There are 
mixed results among high performing second and third grade students. While the 
proportion of high performers increased in first grade in 2001, the percentages fluctuated 
in second and third grade during previous years. 

Longitudinal Analysis of Reading First Students' Achievement. In the following 
analyses, the ITBS performance of students who have participated in the Reading First 
program for the full three years (1999, 2000 and 2001) are examined. These students 
were in first grade during the first year of implementation and in kindergarten in 1998, 
the year in which pre-implementation ITBS data were collected. Kindergarten students 
are not required to take the ITBS as part of the program, therefore, pre-implementation 
data are not available for this cohort of students. Table 3 below shows the percentage of 
students whose individual ITBS reading vocabulary and comprehension scores increased 
over the three-year period. These data address the question: “Has the achievement of 
students who have participated in Reading First for the full duration of the program 
consistently increased?” The table below contains the percentage of students whose 
scores increased from year one to year three (1999 to 2001) and, year two to year three 
(from 2000 to 2001). 

Table 3. Percentage of Reading First Students With Increased Comprehension and 



Vocabulary ITBS Scores, 


1999-2001 






1999-2001 


2000-2001 


Reading Vocabulary 


70.5% 


65.1% 


Reading Comprehension 


80.2% 


62.8% 



The data show that students made the greatest gains in achievement when vocabulary and 
comprehension scores were examined from year one (1999) to year three (2001). The 
vast majority of students (70.5 and 80.2 percent) had higher scores in vocabulary and 
comprehension respectively from 1 999 to 200 1 . Likewise, more than half of the students 
who participated in Reading First for three years had higher vocabulary and 
comprehension scores when 2000 ITBS data are compared to 200 1 . It is important to 
point out that in year one of the program (1999), implementation was poor, therefore, 
large gains in achievement would be expected from 1 999 to 200 1 which compares little 
or no implementation to two years of full implementation. The fact that scores also 
increased from 2000 to 2001 (year two to year three) when the program was fully 
implemented in the Reading First schools is even more important. These data show, for 
example, that while 70.5% of schools increased their vocabulary scores from 1999 to 
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2001, 65.1% of this increase may have occurred during year two of the program (2000 to 
2001). This suggests that achievement has improved as a result of properly implementing 
the Reading First program. 

Criterion-Referenced Competency Scores for Reading First Schools. During the 1999- 
2000 school year, the GDOE began administering the CRCT to students in grades four, 
six, and eight. The CRCT is closely aligned to the Quality Core Curriculum (QCC) and, 
therefore, is an appropriate indicator of student performance in each of its content areas. 
This section of the evaluation examines the reading performance of fourth grade students 
who attended Reading First schools using the CRCT. Even though the Reading First 
program serves students in grades K through three, the CRCT can be used as a means to 
assess the long term effectiveness of the program. The data presented here compare the 
performance of students who attended Reading First schools during the first and second 
years of implementation. In other words, the 2000 CRCT scores are for students who 
were in grade three during the program’s first year of implementation (1998-1999) and in 
grade four during the 2000 administration of the CRCT. Likewise, the 200 1 CRCT data 
are for students who were in grade two during the first year of implementation, and grade 
three during the second year of Reading First. These students were in fourth grade during 
the 2001 administration of the CRCT. The purpose of this comparison is to assess the 
extent to which longer participation in the program (one versus two years) results in 
higher reading achievement on the CRCT. 

Chart 7, which is on the following page, compares the CRCT performance levels (does 
not meet the standard, meets the standard, exceeds the standard) of fourth graders at 
Reading First schools. These data show that when compared to 2000, fourth graders 
improved their reading performance in 2001. More specifically, the percentage of low 
performing students (those not meeting the standard) decreased by 10-percentage points 
in 2001. Conversely, there was also a 10-percentage point gain in the number of students 
meeting or exceeding the standard in 200 1 when compared to 2000. Again, these data are 
consistent with previous findings that indicate that the Reading First program was not 
fully implemented during year one (1998-1999) and has had little effect on third grade 
students. However, students with two years of participation (second and third grades) 
had better performance on the CRCT. 
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Chart 7. Percentage of Fourth Grade Reading First Students Scoring at Each 
Performance Level 
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Even more importantly, of the 302 Reading First schools that serve fourth grade and have 
2 years of CRCT data, 83 percent showed increases in the percentage of students meeting 
or exceeding the standard from 2000 to 2001 . The analyses presented here show that 
students who attended Reading First schools in second and third grade performed better 
on the CRCT than those who attended Reading First schools for third grade only. The 
individual CRCT scores for Reading First schools are provided in Appendix D. 

Stanford-9 Scores for Reading First and Non Reading First Schools. In the spring of 
2001, the Stanford-9 (SAT-9) norm-referenced test was administered to students in 
grades three, five, and eight for the first time as part of the state’s testing program. The 
SAT-9 replaced the ITBS that had been in use in previous years. This section of the 
evaluation presents third grade reading SAT-9 scores for a comparable sample of 
Reading First and non Reading First schools (n=200). The sample was selected and 
matched based on the following criteria: 1 ) achievement using their 2000 reading 
comprehension ITBS scores, and 2) socio-economic status using the percentage of 
students eligible for free and/or reduced lunch. The purpose of this section of the report is 
to compare the reading achievement of students attending Reading First and non Reading 
First schools. This sections answers the question: “Do Reading First students perform 
better or worse than their peers enrolled at schools with similar demographics where a 
different reading initiative is in place?” 

Several analyses are presented here. First, 2001 vocabulary and comprehension SAT-9 
scores for the sample are compared. Next, a comparison of the proportion of Reading 
First and non Reading First students scoring in the upper and lower quartiles of the SAT- 
9 is made. Finally, the percentage of Reading First and Non Reading First schools whose 
scores have improved from 2000 to 2001 is presented. The data used for this analysis is 
the 2000 ITBS and 2001 equated Stanford-9 scores. The equated score, as provided by 
Harcourt Educational Measurement, estimates what Georgia’s student percentile ranking 
scores would have been had students taken the ITBS instead of the Stanford 9. 

Chart 8 shows the 2001 SAT-9 reading vocabulary and comprehension percentile scores 
for the sample of Reading First and non Reading First students. The data indicate that 
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there is a slight difference in reading achievement between the two groups. Reading First 
students had higher vocabulary and comprehension scores (1 percentile point) than their 
non Reading First peers. 

Chart 8. Comparison of 2001 SAT-9 Scores for Reading First and Non Reading 
First Students 



100 




Vocabulary Comprehension 



□ Reading First 
BNon Reading First 



Table 4 contains the percentage of Reading First and non Reading First students scoring 
in the lower and upper quartiles (at or below the 25 <h percentile and at or above the 75 th 
percentile) on the SAT-9. The purpose of this comparison is to assess whether there are 
differences in the proportion of low and high achieving students between the two groups. 
The data show that in both content areas, there were fewer Reading First students scoring 
at or below the 25 th percentile than non Reading First students. However, more non 
Reading First students scored at or above the 75 lh percentile than Reading First students. 
In other words, these data suggest that there are fewer low performing third grade 
Reading First students than non Reading First students. Additionally, non Reading First 
schools had more high performing students (at or above the 75 th percentile) than Reading 
First schools. These data are consistent with the finding that Reading First is effective 
with low performing third grade students. This finding also lends support to the previous 
evaluation recommendation which suggested that the Reading First program needed to 
become more rigorous, particularly for average and high performing second and third 
graders. 
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Table 4. Percentage of Third Grade Reading First and Non Reading First Students 
Scoring At or Below the 25 th Percentile and At or Above the 75 th Percentile, 2001 





Vocabulary 


Comprehension 


Low Achieving Students 
(At or Below the 25 th Percentile) 

Reading First 


26.8% 


32.7% 


Non Reading First 


30.4% 


35.7% 


High Achieving Students 
(At or Above the 75 th Percentile) 

Reading First 


14.9% 


18.6% 


Non Reading First 


16.2% 


20.5% 



The last comparison using the SAT-9 answers the question: “Have Reading First schools 
shown greater overall improvement in reading when compared to their non Reading First 
peers?” Using the reading comprehension ITBS (2000) and SAT-9 equated scores 
(2001), the percentage of Reading First and non Reading First schools that had increased 
third grade scores was calculated and is presented in Table 5. These data show that more 
Reading First schools increased their reading comprehension scores than their non 
Reading First peers. Appendix E contains the SAT-9 scores and selection criteria for the 
sample of Reading First and non Reading First schools. 

Table 5. Percentage of Reading First and Non Reading First Schools With 
Increased Third Grade Reading Comprehension Scores, 2000 -2001 
Reading Comprehension 

Reading First 47% 

Non Reading First 41% 

Summary on Achievement 

Overall, the achievement data presented support the findings from the year one and year 
two evaluations. The data show that the Reading First program benefits first grade 
students and low achieving second and third grade students. While the Reading First 
program was revised to provide more rigorous third grade instruction, the effects of the 
new comprehension module are not apparent. Like the Reading First program itself, the 
comprehension module may take more than one year to fully implement or measure. 

This section of the report focused on the academic achievement of Reading First students. 
The data support previous evaluations and the continued use of the Reading First 
program. The type of instruction required as part of the program clearly benefits first 
grade students. Even though the effects of the new component (comprehension module) 
on second and third grade students are not clear, it is important to continue examining 
ways to provide better instruction and to fully impact the targeted student population. 
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Part III: Program Implementation and Achievement 



The last section of this report focused specifically on the reading achievement, as 
measured by the ITBS, SAT-9 and CRCT scores of Reading First students. This section 
of the report examines reading achievement within the context of program 
implementation - what teachers and administrators did in schools that might account for 
student performance. The data presented in this section were obtained through discussion 
groups and surveys with Reading First and non Reading First teachers and administrators, 
as well as surveys with Reading First and non Reading parents. Copies of the discussion 
group protocols and survey instruments can be found in Appendix B. The full text of the 
discussion group report will be made available upon request. 

This section contains the following sub-sections: 

• Philosophy of Teaching Reading 

• Reasons for Implementing Reading First or Other Reading Programs 

• Satisfaction with Reading First 

• Parental Involvement 

• Staff Development 

• Indicators of Academic Success 

• Changes in Reading First and Other Reading Program Over the Three Year 
Period. 

• Future Plans for Reading 

• Recommendations for Improvement 

Philosophy towards teaching reading. Participants across all discussion groups 
(Reading First teachers, non Reading First teachers, and administrators) stressed similar 
ideas when talking about their philosophy towards teaching reading. Both Reading First 
and non Reading First teachers shared the view that “ reading is the foundation for all 
learning and should be the focus of instruction in the primary grades .” For example, a 
second grade teacher stated that " social studies, science, and math should be added to the 
curriculum only after students have learned to read and comprehend. ” 

Participants also stated that primary-level teachers must ensure that every child leams to 
read and encouraged the use of varying instructional practices to do so. For instance, 
participants agreed that “all children can learn to read if they are taught in the way they 
need to be taught. ” To that end, teachers felt that they need to be able to accurately 
assess students’ reading skills and provide the necessary intervention strategies. Reading 
First teachers considered the Basic Literacy Test (BLT) to be an effective tool for 
diagnosing literacy skills. Most of the teachers also believed that schools must adopt a 
" balanced approach ” to reading which emphasizes whole language instruction, phonics, 
and comprehension. Non Reading First teachers indicated that the reading programs in 
their schools incorporated different approaches to teaching reading including whole 
language and phonics. Several of the non Reading First schools indicated that they used 
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the Reading Recovery, Literacy Collaborative, and Success for All programs in lieu of 
Reading First. 

Teacher’s orientation toward teaching reading was examined as part of the teacher survey 
that was administered to a sample of Reading First and non Reading First teachers in the 
spring of 200 1 . The survey, developed by researchers at Ohio State University, is based 
on the assumption that teachers’ reading orientation establishes expectancies and 
influences goals, procedures, materials and classroom instructional practices. Chart 9 
compares the percentage of Reading First and non Reading First teachers whose 
responses to the survey items were categorized as one of the following three reading 
orientations: 1) Phonics (isolation of phonemes - emphasis on decoding), 2) Skills 
(isolation of skills - emphasis on words), and 3) Whole Language (no isolation of skills 
for practice - emphasis on developing sense of story and text). 

Chart 9. Comparison of Reading First and Non Reading First Teachers’ Reading 
Orientation 



100 
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These data show that both Reading First and non Reading First teachers had a strong 
orientation towards the provision of skills when teaching reading. Approximately one- 
quarter of teachers’ orientation was towards phonics and an even smaller percentage 
focused on whole language. 

Administrators for both Reading First and non Reading First program shared similar 
views about teaching reading. Many said that “teachers must do whatever it takes to help 
students learn to read. ” Administrators indicated that teachers must be willing and able 
to address “different learning modalities” and to form “appropriate” expectations of their 




3 

ERIC 



Georgia Department of Education 
Linda C. Schrenko, State Superintendent of Schools 
November 200 1 • Page 20 of 42 
All Rights Reserved 



26 



students. For example, one administrator said that, "If we can ’t teach children at their 
own level, we are destined for failure. ” Administrators also stressed that teachers need to 
be creative, but cautioned that creativity should be guided by the Quality Core 
Curriculum (QCC). 

Reasons for Selecting Reading First or Other Reading Programs. Teachers and 
administrators were asked why they chose to implement the Reading First program or 
other reading programs to meet their student’s needs. Many Reading First stakeholders 
indicated that their schools implemented the Reading First Program because of the 
“ additional funding” that the program provided. They saw the additional resources as a 
way to “level the playing field” with other schools. Given that the demographic data on 
Reading First schools show that a large percentage have low SES this information is not 
surprising. Several discussion group participants also stated that “ Reading First was 
selected because it provides a “solid foundation for reading at the primary level' that can 
be reinforced in the upper grades. " 

Non Reading First personnel provided several reasons why their schools did not 
participate in the Reading First Program. Some non Reading First staff felt that their 
schools already had an effective reading program in place. Many participants felt that 
“the Reading First program requirements were too restrictive and that Reading First did 
not provide an adequate focus on writing. ” A vast majority of non Reading First staff 
felt that “ the Reading First program required excessive testing and that the three-hour 
block of reading instruction that is required was unnecessary .” Several administrators 
indicated that their schools’ test scores were sufficiently high with their existing reading 
program, and therefore, there was no need to implement something new. Others 
suggested that Reading First had not been “proven consistently” and that the money 
awarded to implement the program came with “ too many strings attached. ” 

Non Reading First teachers and administrators that were in favor of the program felt that 
it would be too difficult to implement without paraprofessionals. Others simply said that 
they had applied for the grant but were denied. 

Satisfaction with the Reading First Program. For the most part, Reading First teachers 
and administrators agreed that Reading First met or exceeded their expectations. One 
teacher stated that, “ the state finally realized that in order to teach science and social 
studies, students need to be able to read first. ” Teachers and administrators further 
agreed that “ the phonics component of Reading First had resulted in earlier literacy 
among K-2 students.” Some administrators believed the “ amount of teamwork and 
dedication required to implement Reading First had improved the overall climate of their 
schools. ” 

In spite of a consensus that Reading First met or exceeded most expectations, some 
participants expressed concerns that the BLT was too subjective to reliably and validly 
assess students’ needs. There was also concern that the BLT would be used 
inappropriately to measure teacher effectiveness instead of students’ literacy skills. 
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Reading First administrators thought the BLT was an effective tool for assessing student 
strengths and weaknesses, but expressed concerns that students were being tested too 
much, that teachers lost too much instructional time meeting testing requirements, and 
that teachers were not adequately trained to score the BLT. Teachers also commented 
that Reading First materials were not available initially (as documented in the year one 
evaluation) because of funding delays at the state and district levels. Several 
administrators found the state-imposed “letter of assurance” to be professionally 
insulting. 

Reading First and non Reading First alike reported that the greatest strength of any 
instructional program is the availability of funds and materials that support the programs. 
For instance, Reading First teachers said that they found the phonics component, the BLT 
and the continuity of the program across and within grade levels to be valuable assets. 
Administrators noted that the primary strength of the reading programs were the 
“ teachers who worked extremely hard and knew the importance of reading instruction in 
the primary grades .” 

Comparisons of Reading First and Other Reading Programs. The Reading First 
program requires three hours of reading instruction each day. Some schools indicated 
that they were able to set aside one uninterrupted block of time for Reading First, 
whereas others had to schedule shorter periods throughout the day. Despite scheduling 
differences, all teachers reportedly incorporated the required components into their 
curricula. Many of the non-Reading First teachers reported that their schools had 
implemented reading programs that contained some of the Reading First components. 

For example, some non Reading First schools incorporated phonics, small and whole 
group instruction, and writing into their reading programs. Many non Reading First 
teachers indicated that they used the BLT to assess students’ reading abilities, while 
others used STAR Assessment, Developmental Reading Assessments (DRA), and 
Scholastic Reading Inventory (SRI). Some non Reading First schools also selected well- 
known reading programs such as Success for All (which is also used at some Reading 
First schools) and Reading Recovery, while others implemented school-based or teacher- 
designed programs. One teacher’s school reportedly used a “hodgepodge of reading 
activities with little or no continuity within or across grade levels ” while others agreed 
that administrators “ don ’t really care what we do in the classroom as long as the kids 
show effort on the reading test. ” Participants from all groups reported that most new 
teachers were not prepared to teach reading, especially phonics. 

The survey administered to the sample of Reading First and non Reading First teachers 
gathered information on the types of instructional practices used in the classrooms and 
the amount of time spent on components that are central to Reading First and other 
reading programs. When asked about the goals of their reading programs, more than 
seventy percent of Reading First and non Reading First teachers agreed that teaching 
phonics and sight words were central to their reading instruction, while approximately 
half of teachers indicated that the use of computers to reinforce or manage reading was a 
major part of their program’s goals. Compared to non Reading First teachers, a greater 
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percentage of Reading First teachers reported having the three components (phonics, 
sight words and computers) as part of their program’s goals. Chart 10 compares the 
percentage of Reading First and non Reading First teachers who indicated that the use of 
phonics, sight words and computers were central to their reading program. 
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Chart 10. Percentage of Reading First and Non Reading First Teachers Reporting 
Phonics, Sight Words and Computer Usage as Program Goals 




Phonics Sightwords Computers 



& Reading First 
UNon Reading First 



Chart 1 1 compares the amount of time Reading First and non Reading First teachers 
spent on each of the following instructional areas: phonics, sight words, small group 
instruction, computer usage. These data show that there are slight differences between 
Reading First and non Reading First teachers’ classroom practices. Using a scale of one 
through five (1 = 0-20 minutes, 2 = 21-40 minutes, 3 =41-60, 4 = more than 80 minutes), 
the data show that on average, Reading First teachers spend a little more time each day 
teaching phonics, sight words, using small group instruction and using computers to 
reinforce and manage reading instruction than non Reading First teachers. For instance, 
the average score of Reading First teachers was 2.5 for the amount of time spent teaching 
phonics compared to 2.0 for non Reading First teachers. 

Chart 1 1 . Average Score for the Amount of Time Spent on Various Instructional 
Practices 
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The discussion group data also indicated that computer use varied in both Reading First 
and non Reading First schools from no use at all to approximately twenty minutes per 
day, and was largely dependent on availability. Many schools with Reading First or other 
reading programs used computers to reinforce reading through the Accelerated Reader 
program. One school used Reading First funds to pilot Breakthrough to Literacy, a 
program which requires students to spend 20 minutes per day using computers to work on 
reading. Some of the non Reading First teachers reported that they were required to 
create web pages and Power Point presentations on the computers and used them 
minimally for student instruction. Administrators across the groups reported that most 
schools had computer labs and classroom computers available for use. 

Teachers and administrators were asked in the discussion groups how instruction differed 
by grade level. Reading First and non-Reading First teachers indicated that they focused 
primarily on phonics and sight words from Pre-K to grade one, and changed the emphasis 
to comprehension by grades two or three. Both sets of third grade teachers usually 
focused on grammar and spelling, while teachers in the higher grades tended to focus on 
“reading to learn” rather than “learning to read.” Others added that research skills are also 
introduced in grade one so that students will be able to complete a full research paper by 
grade five. 

Administrators and teachers in both Reading First and non-Reading First schools stated 
that their schools integrated reading across the curriculum. Many felt that the three-hour 
literacy requirement of Reading First necessitated reading in the other subject areas. In 
some Reading First schools, subject area teachers were trained to assist Reading First 
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teachers whereas in other schools, subject area teachers integrated the Reading First 
strategies into their own classes. 

Non Reading First schools that implemented Success for All trained all teachers in the 
reading process and actively involved them in teaching reading. Although most reading 
teachers reported that they had met and planned thematic units with the subject area 
teachers, some teachers reported that science and social studies were not fully addressed 
in their schools because of the amount of instructional time that had been set aside for 
teaching reading. 

Parental Involvement. Reading First and non Reading First discussion group 
participants all agreed that, overall, parental involvement needs to be improved. Although 
school personnel reported encouraging parents to attend workshops and other events at 
the school, attendance usually tended to be low. In some cases, teachers reported that 
when incentives such as food and prizes were available, attendance increased. However, 
for the most part, participants reported that parents tended to be more involved in 
student’s take home activities such as vocabulary lists, sight words on flash cards or 
reading together with their children each night. 

Several teachers in the discussion groups (both Reading First and non Reading First) said 
that they kept parents involved in the reading process by requiring them to “sign off that 
the student had completed home activities and/or that they reviewed materials and reports 
sent home. While most teachers reported sending home reading progress reports and 
telephoning parents to discuss student performance, some set up web pages with class 
news and information. 

Many Reading First and non Reading First teachers were concerned that some parents 
lacked the necessary skills to help their children learn to read. A few teachers noted that 
they “ try to work around these parents and provide opportunities to assist them. " Some 
of the initiatives that schools implemented to increase parent involvement were: 

• Family Reading Nights 

• Literacy Partners (teaching parents how to help students with reading) 

• America Reads (sponsored through a local university) 

• VISTA volunteers 

• Church volunteers 

. Reading Is Fundamental (RIF) 

• Fathers Being Involved (FBI) 

Teachers reported that some parents were learning along with their children because they 
had never had phonics instruction. Some parents even considered the phonics 
components to be the selling point of the reading program. Conversely, parents of 
students who struggled with reading thought the programs were too hard for their 
children. 
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As part of this final evaluation, surveys were administered in the spring of 2001 to 
approximately 3,000 parents with students attending Reading First and non Reading First 
schools. Overall, the data show that Reading First and non Reading First parents 
participate in their child’s school at approximately the same rate. For instance, Chart 12 
shows the percentage of parents who work outside of the home and who belong to 
Parent-Teacher Association (PTA/PTO) at their child’s school. These data suggest that 
there is very little difference between Reading First and non Reading First parents on 
these dimensions. Nearly seventy percent of parents work outside of the home and 
approximately sixty percent are members of the PTA or PTO. 

Chart 12. Percentage of Reading First and non Reading First Parents Employed Outside 
of the Home and Belonging to Parent Groups 
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Chart 13 shows the percentage of parents who reported participation in school activities, 
volunteering at their child’s school, and talking with the teacher regularly and receiving 
written communication (other than report cards) about their child’s academic progress. 
Again, these data show little difference between Reading First and non Reading First 
parents. The vast majority of parents indicated that they participate in school activities 
(92%) and talk with their child’s teacher (96% and 98%, respectively). Over seventy 
percent of parents reported that they volunteer in their child’s school, with Reading First 
parents indicating that they had slightly higher rates of volunteerism (73% vs. 71%). 
Compared to Reading First parents, a larger percentage of non Reading First parents said 
that they received written communication (80% vs. 73%) from their child’s teacher about 
his/her academic progress. 

Chart 13. Percentage of Reading First and non Reading First Parents Involved in 
their Child’s School 
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On the surveys, parents were also asked about the types of educational activities they 
engaged in at home. Using a five-point scale, parents were asked to indicate how often 
(e.g.; l=less than once a month, 5= almost every day ) they engaged in activities that 
enhanced their child’s education. Table 6 contains the average response (using the five 
point scale) of parents on reading related activities. These data also show that there was 
very little difference in the responses of Reading First and non Reading First parents on 
the dimensions measured. For the most part, both Reading First and non Reading First 
parents indicated that they spend at least one day each week (rating > 4) on each of the 
listed activities. Parents spent the most time talking with their child about what s/he is 
learning in school, listening to their child read or attempt to read, and checking or 
providing help with homework. Admittedly, parents only visited the library and brought 
home reading materials one time per month or less. 
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Table 6. Comparison of Parents’ Average Ratings for Participation in Educational 
Activities at Home 



How often do you... 


Reading First 


Non Reading 

First 


talk with your child about what s/he is 
learning 


4.8 


4.8 


read aloud to child 


4.0 


4.2 


listen to your child read or attempt to 
read 


4.5 


4.5 


help or encourage child to write 


4.4 


4.4 


check or provide help with homework 


4.7 


4.8 


Child tries to read on his or her own 


4.6 


4.6 


talk with your child about a book s/he 
is reading 


4.4 


4.5 


visit the library with your child 


2.4 


2.5 


bring home books or other reading 
materials for your child 


2.9 


3.0 



Table 7 below shows the percentage of parents that engage in specific activities geared 
towards helping their child to read. These data show that across grade levels, most 
Reading First and non Reading First parents helped their child by sounding out words, 
listening to him or her read, and identifying words. In almost all cases, a larger 
percentage of non Reading First parents spent time on these activities than Reading First 
parents did (except for second grade where a larger percentage of Reading First parents 
listened to their child read). It may be that given the heavy emphasis on these activities at 
Reading First schools, slightly fewer parents felt like their child needed these types of 
assistance. 



Table 7. Percentage of Parents Providing Reading Assistance by Grad e 





1 st Grade 




2 nd Grade 




3 rd Grade 






Reading 


Non 


Reading 


Non 


Reading 


Non 




1st 


Reading 

1st 


1st 


Reading 

1st 


1st 


Reading 

1st 


Identify 

words 


60% 


68% 


56% 


56% 


59% 


62% 


Help with 
sounding out 
words 


81% 


83% 


69% 


75% 


64% 


65% 


Tell letter 
names 


19% 


24% 


12% 


17% 


12% 


13% 


Tell sounds 
of letters 


36% 


46% 


26% 


38% 


22% 


25% 


Listen to 
him/her read 


74% 


84% 


83% 


78% 


78% 


79% 
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Overall, parents reported that they engaged in activities that were supportive of the 
reading programs employed in their child’s school. Although teachers and administrators 
felt that parental involvement was low overall, they did indicate that parents were more 
likely to participate in activities at home than at school. The survey data support this 
notion. Parental involvement has consistently been an issue of concern to schools. 
Parents, administrators, and teachers would benefit from working cooperatively to better 
meet student’s needs. 

Staff Development. When asked about staff development opportunities, Reading First 
teachers admitted that they were generally satisfied with the training and support they 
received as part of the program Most stated that they were shown how to effectively 
organize reading instruction across grade levels, how to use the BLT and LEXIA to 
assess student needs, and how to teach phonics and reading comprehension. While 
many Reading First teachers thought the 50-hour training requirement for the first year 
was appropriate, they also felt the 30-hour requirement for subsequent years was 
“overkill.” For example, some teachers thought that state trainers need to come from the 
“classroom ” instead of the office and that training would be more effective if it were 
more “hands-on. " Many teachers also agreed that “ teachers need to learn different 
reading strategies for students in grades three through five, and that the Reading First 
program should incorporate more writing across the curriculum. ” Finally, most 
Reading first teachers agreed that many new teachers have had no training in teaching 
reading and, therefore, that the Reading First training can be overwhelming. 

Teachers at non Reading First schools, specifically those that implemented the Literacy 
Collaborative, Success for All and Reading Recovery programs, expressed an overall 
satisfaction with their training and support services. However, they were concerned that 
staff training was typically a one-day session on “everything”, trainers were often not 
qualified or merely read notes, and that there was no release time for the training. 

Administrators across the board shared the teachers’ concern that new teachers often did 
not know how to teach reading. For instance, one administrator stated that, "many 
teachers do not know how to teach comprehension in Grades K-5. ” As a remedy, the 
administrators recommended that each school acquires a literacy trainer who can provide 
ongoing support and training. Administrators at Reading First and non Reading First 
schools also believed that “all teachers could benefit from additional training in student 
assessment; technology for instruction, writing, phonics. " 

Indicators of Academic Success. Both Reading First and non-Reading First teachers 
thought that their respective reading programs were very effective. They noted that 
students were reading earlier, they were more excited about reading, and parents were 
generally pleased with the progress of their children. They also observed students reading 
more in class. In terms of assessing students’ progress, both Reading First and non 
Reading First teachers admitted that they administered self-developed reading inventories 
and collected writing samples from students. While Reading First teachers relied more 
heavily on the BLT to assess reading progress, some non Reading First teachers reported 
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that they used the GKAP, ITBS, SAT-9, Diagnostic Reading Assessment (DRA), STAR 
Assessment for Accelerated Reader and Scholastic Reading Inventory (SRI) to monitor 
student’s progress. Teachers and administrators across both groups also agreed that the 
CRCT would be a more accurate assessment of reading progress than the norm- 
referenced tests. 

Teachers and administrators in both groups also reported that more students were excited 
about reading. As one teacher put it, “the more they read, the more they enjoy it. " 
However, some teachers were concerned that a few students could not identify with the 
content of classroom reading materials. This was especially true for ESOL students, who 
often did not experience the same level of success with reading as their non-ESOL 
classmates. 

Changes in Reading First and Other Reading Programs Over the Three-Year Period. 
Overall, teachers and administrators reported that there had been few changes in their 
reading programs over the past years. However, Reading First teachers noted that the 
biggest differences from year one of the program to the present were changes in attitude 
about teaching reading. Reading First teachers reported that they were "more confident 
about teaching reading as a result of working with Reading First, and were more 
comfortable implementing components of the program such phonics instruction, BLT 
assessment, and literacy centers. " Participants from both Reading First and non Reading 
First schools expressed concern that paraprofessionals were no longer available to assist 
many classroom teachers. 

Future Plans for Reading. Almost all of the Reading First teachers thought their schools 
would continue the program in or near its original form during the next school year. A 
few teachers indicated that their schools had planned to acquire more appropriate 
materials for students in grades four and five. Teachers from one district stated that their 
system was switching to Success for All in the 2002-2003 school year as part of a 
district-wide school reform initiative. Nearly all of the non-Reading First teachers 
reported that they would most likely continue with the programs they had during the next 
school year without any major changes. 

Many administrators in both groups expressed frustration that they did not know what the 
state’s plans were for funding reading programs in the upcoming school year. Some 
administrators noted the need for more research about which reading programs really 
worked and under what conditions they were most useful. Most of them, however, 
indicated they had planned to continue with their existing reading program. 

Even though Reading First and non Reading First teachers thought they would continue 
with their existing reading programs and generally agreed that their programs adequately 
met the needs of their students, there were still some concerns. For instance, both 
Reading First and non-Reading First teachers thought their programs worked well for 
average students but did not serve the “lowest” and “highest” students very well. 
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Teachers also expressed concern that neither Reading First or other reading programs 
being used adequately met the needs of ESOL students. 

Recommendations for Improvement. Most of the participants in the Reading First and 
non Reading First discussion groups offered recommendations that they believe would 
improve reading in Georgia. The recommendations centered mainly on funding, 
evaluation of existing reading programs, student assessment, continuity across the 
curriculum, staff development, and parent involvement. The recommended areas of 
improvement have been identified throughout this part of the report and are summarized 
below: 



• Increase state and local funding to support reading instruction in all schools 

• Use curriculum based assessment tools for determining student progress 

• Use both state tests and school based indicators of success to determine the 
effectiveness of reading programs 

• Improve the continuity of reading programs across grade levels 

• Increased focus on instruction in writing 

• Provide more staff development and ongoing support in the school 

• Provide more training and support in reading instruction for all new teachers 

• Reduce class size and/or add paraprofessional assistance to allow for more small 
group or individual instruction 

• Align instruction with state QCC, district requirements 

• Address the special needs of ESOL students and students with learning 
disabilities 

• Increase parent involvement 



Reading First staff also recommended that the following issues be specifically addressed 
to improve the Reading First program: 

• Expand the approved list of materials and resources for Reading First funds 

• Expand the Reading First program and staff development opportunities to grades 
four and five 

• Revise the BLT 

• Provide additional staff development related to administration and scoring of the 
BLT and its use in planning for instruction 

• Improve of state-level training and the re-delivery system for Reading First 
training at the local level 

Summary on Implementation 

The implementation data colleted in this evaluation show that Reading First and non 
Reading First teachers are share similar philosophies towards teaching reading, 
classroom instructional practices and concerns about parent involvement and staff 
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development. The data presented here suggest that the components of the Reading 
First program are included in other reading initiatives and are widely endorsed by non 
Reading First schools. Teachers, administrators and parents alike are all committed to 
providing quality reading instruction, and supporting the instruction that best meets 
students’ needs. 
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Part IV: Summary and Conclusions 

The results from the final evaluation of the Reading First program support the findings 
from previous years and the continued use of the Reading First program. The data from 
all three of the standardized tests used to assess student achievement (ITBS, CRCT, SAT- 
9) support one another. The achievement data suggest that the Reading First program is 
most beneficial for first graders and low achieving second and third grade students. 

There is no evidence that the program significantly affects the achievement of high 
achieving second and third graders. It is too soon to measure the effect of the 
comprehension module that was developed and implemented during the 2000-01 school 
year to better address the needs of second and third grade students. 

The discussion group and survey data showed that Reading First and non Reading First 
teachers and administrators share similar philosophies towards teaching reading and 
employ similar instructional techniques in the classroom. Both Reading First and non 
Reading First parents report engaging in activities that clearly support reading in general, 
and, Reading First, in particular. 

In terms of program improvement, several themes emerged among Reading First and non 
Reading First staff. The main areas of concern are regarding continued funding, improved 
staff development and continuity across the curriculum. Addressing the aforementioned 
needs could further enhance the Reading First program as well as other reading programs 
being used in Georgia’s elementary schools. 
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Appendix A 



Reading First Guidelines 
2000-2001 



I. ADVISORY COMMITTEE 

A. Establish an advisory committee consisting of: 

1. Principal - Chair 

2. Teacher representatives from all grades in the school 

3. Representatives from special education, ESOL, Title I, and E1P 

4. Media specialist 

5. Central Office leadership 

6. Parent volunteer and/or community leader 

7. RESA consultants and Field Service Director (if possible) 

B. Convene a minimum of two advisory committee meetings during the year. 

II. READING INSTRUCTION 

A. Focus on reading instruction for at least three hours each day in grades K-3. 

The three hours will be divided to include instruction using: 

1. Direct, systematic, explicit phonics (Only an approved phonics program 
may be used. Refer to the Reading First Instructional Materials List.) 

2. Quality literature (Basal reading programs with quality literature would 
meet this requirement.) 

3. High frequency sight vocabulary (Refer to the Reading First Instructional 
Materials List for material that may be used for instruction in high 
frequency sight vocabulary.) 

4. Teaching reading within the content areas which may include thematic 
teaching of reading addressing the QCC requirements at each grade level 

5. Reading comprehension strategies from the summer training and the 
Comprehension Manual should be incorporated in each of the above. 

B. Students will read at least 25 books during the year. A computer program should 
be used to assist teachers in monitoring this reading. 

III. READING TUTORS 

A. All certified staff in the school not teaching reading will select one child and 
provide one-on-one tutoring in reading for 15-30 minutes twice a week. 

B. Parents and community volunteers are encouraged to participate in weekly 
tutoring sessions and staff development training. 

IV. STAFF DEVELOPMENT 

A. All personnel involved in the Reading First project will complete 30 hours 

of staff development during the school year. The majority of these hours should 
focus on the improvement of instruction to enhance reading comprehension. 

B. The Advisory Committee will participate in planning redelivery of Reading First 
comprehension information to be done by the school representatives who were 
trained during the summer training session. 
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C. The 30 hours of staff development may include: 

1 . Formal redelivery by the school representatives trained during the summer 
using videos, transparencies or Power Point presentations provided by the 
DOE 

2. Ongoing weekly and/or monthly grade-level or whole-faculty meetings for 
the development and implementation of strategies 

3. Training provided by publishers for new materials 

4. Study groups convened for evaluating current practices in light of new 
research 

D. Schools will provide a Reading First Comprehension Manual for all staff 
participating in the project. This manual may be copied in the school or 
district office. 

E. Grant funds may be used for teacher stipends for off-contract work time, not 
to exceed $4,000. 



ASSESSMENT 

A. The Basic Literacy Test will be individually administered to students in grades K- 
3 by teachers in Reading First schools during the fall, winter and spring. The 
Basic Literacy Test assesses decoding skills, sight words, and reading 
comprehension. Every student participating in the project must be assessed. 
(Students assessed the previous spring are exempt from fall assessment.) 

B. In order to complete the longitudinal study begun in 1998-1999, schools 
participating in the Reading First Project at that time are required to assess all 
students in the first, second, and third grades with the Reading Vocabulary and 
Reading Comprehension portions of the Iowa Tests of Basic Skills. This will be 
in addition to mandatory testing in third grade with the Stanford 9. 

C. The ITBS is not required for kindergarten students, however, if a school chooses 
to administer the ITBS in kindergarten, Reading First funds may be used to 
purchase test materials. 
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Appendix B Survey Instruments and Discussion Group Protocols 

Parent Survey 



Spring, 2001 




The Georgia Department of Education (GDOEI is in the process of learning about re ading programs 
in the elementary schools. We recognize the importance of parental involvement in education and 
ask that vou complete the following survey. Your answer s will provide valuable information about 



the types of activities vou participate in with vour child and the degree to which vou are satisfied 
with vour school's reading program. Your answers will remain anonymous— do not write your 
name on this survey. You mav fill in vour responses in pencil or pen. For vour convenience, we 



have provided a self-addressed stamped envelope. Please return vour survey to: Dr. Valerie Roberts. 
Georgia Department of Education. 1754 Twin Towers East. Atlanta. GA 30334 by June 30. 2001. If vou have 
anv questions, please contact Dr. Valerie Roberts at (4041 656-2668 or via email at 
vroberts@doe.kl2.aa.us. 





K 


1 


2 


3 


1. What is your child's grade level? (If you have more 
than one child in grades K through 3, please fill out this survey 
for only one child.) 


□ 


□ 


□ 


□ 




Yes 


No 


2 Are you employed outside the home? 






□ 












□ 








3. Do you belong to a parent group (for example, PTA 
or PTO) at your child's school? 






n 












□ 








The following questions ask you to indicate the extent to which 
you participate at your child's school and communicate with 
teachers about your child's education. Please respond by 


Yes 


No 


How often does this occur? Please 
respond below only for the questions to 
which you answered 'yes' 


marking 'yes' or 'no' in the boxes to the right. If you answer 
'yes' to any of the questions, please indicate how often these 
activities occur by marking the boxes to the far right. 






Once a 
year 


Once 

every 

few 

months 


Once a 
month 


At least 
once a 
week 


Every 

day 


4. Do you volunteer at your child's school? 


□ 


□ 


□ 


n 


□ 


□ 


□ 


5. Do you participate in or attend activities at your 
child's school? 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


6. Do you talk with your child's teacher about his or 
her academic progress? 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


7. Do you receive written communication (other than 
report cards or deficiency reports) from your child's 
teacher about his or her academic progress? 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


Please tell us how often you participate in the following 
activities with your child. 


Less 
than 
once a 
month 


Once a 
month 


Several 
times a 
month 


Once a 
week 


Almost 
every day 


8. How often do you and your child talk about what s/he is 
learning in school? 


□ 


□ 


□ 


□ 


□ 


9. How often do you read aloud to your child? 


□ 


□ 


□ 


□ 


□ 


10. How often do you listen to your child read or attempt to 
read? 


□ 


□ 


□ 


□ 


□ 


11. How often do you help or encourage your child to write or 
attempt to write? 


□ 


□ 


□ 


□ 


□ 
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12. How often is your child assigned homework? 


□ 


□ 


□ 


□ 


□ 


13. How often do you check your child's homework or provide 
help with homework? 


□ 


□ 


□ 


□ 


□ 


14. How often does your child try to read (i.e.; books, 
maqazines, newspapers etc.) on his or her own? 


□ 


□ 


□ 


□ 


□ 


15. How often do you talk with your child about books s/he is 
reading? 


□ 


□ 


□ 


□ 


□ 


16. How often do you visit the library with your child? 


□ 


□ 


□ 


□ 


□ 


17. How often do you bring home books or other reading 
materials for your child? 


□ 


□ 


□ 


□ 


□ 




Identify 

words 


Help with 
sounding 
out words 


Teii 

letter 

names 


Tell 

sounds 

of 

letters 


Listen to 
him/her 
read 


18. When you help your child read, what kind of help do you 
usually give? Please mark all that apply 


□ 


□ 


□ 


□ 


□ 




Very 

dissatisfied 


Somewhat 

dissatisfied 


Somewhat 

satisfied 


Very 

satisfied 


19. How satisfied are you with the reading program at your 
child's school? 


□ 


□ 


□ 


□ 


20. How satisfied are you with your child's progress in 
learning to read? 


□ 


□ 


□ 


□ 
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Reading First Teacher Survey 
Spring, 2001 




The Georgia Department of Education (GDOE) is completing the final year of the statewide evaluation of the 
Reading First program. This survey is an important part of the evaluation. Your participation is critical to the 
GDOE’s efforts to learn about the Reading First program. Your answers will provide valuable information 
concerning your beliefs about teaching reading, classroom instructional practices and the Reading First 
program components. Your answers will remain anonymous — do not write your name on this survey. You 
may fill in your responses in pencil or pen. To maintain your privacy, please place the survey in the enclosed 
envelope and return it to your school's main office by May 18, 2001. If you would prefer to return your 
survey directly to the GDOE« please mail it to: Dr. Valerie Roberts, Georgia Department of Education. 1754 
Twin Towers East Atlanta^ GA 30334. If you have any questions, please contact Dr. Valerie Roberts at (404) 657- 
6168 or via email at vroberts@doe.kl2.ga.us. 



Please tell us about yourself. 

1 . Which grade is your main teaching assignment? □ K □ 1 □ 2 □ 3 □ 4 □ 5 

2. Which of the following are taught in your classroom during reading (please mark all that apply)? 

□ REP (Remedial Education Program) □ EIP (Early Intervention Program) □ Title I 

3. Did you teach in grades K-3 at this school last year? □ Y □ N 

4. How many years have you taught in grades K-3? □ 1 □ 2 □ 3 □ 4 □ 5 □ 6 □ 7 (D8 □ 9 D10 

□ 11 □ 12 □ 13 □ 14 □ 15 □ 16 □ 17 □ 18 □ 19 □ 20 □ 20+ 

Instructions: Using the scale to the right, please indicate the extent to which you agree with the following 
statements. 



5. Please tell us about your beliefs about 
teaching reading. 


Strongly 

Disagree 


Disagree 


Somewhat 

Agree 


Agree 


Strongly 

Agree 


A child needs to be able to verbalize the rules of phonics in order to assure 
proficiency in processing new words. 


□ 


□ 


□ 


□ 


□ 


An increase in reading errors is usually related to a decrease in 
comprehension. 


□ 


□ 


□ 


□ 


□ 


Dividing words into syllables according to rules is a helpful instructional 
practice for reading new words. 


□ 


□ 


□ 


□ 


□ 


Fluency and expression are necessary components of reading that indicates 
good comprehension. 


□ 


□ 


□ 


□ 


□ 


Materials for early reading should be written in natural language without 
concern for short, simple words, and sentences. 


□ 


□ 


□ 


□ 


□ 


When children do not know a word, they should be instructed to sound out its 
parts. 


□ 


□ 


□ 


□ 


□ 


It is good practice to allow children to edit what is written into their own 
dialect when learning to read. 


□ 


□ 


□ 


□ 


□ 


The use of a glossary or dictionary is necessary in determining the meaning 
and pronunciation of new words. 


□ 


□ 


□ 


□ 


□ 


Reversals (e.g., saying “saw” for “was”) are significant problems in the 
teaching of reading. 


□ 


□ 


□ 


□ 


□ 


It is a good practice to correct a child as soon as an oral reading mistake is 
made. 


□ 


□ 


□ 


□ 


□ 
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It is important for a word to be repeated a number of times after it has been 
introduced to ensure that it will become a part of sight vocabulary. 


□ 


□ 


□ 


□ 


□ 




Strongly 

Disagree 


Disagree 


Somewhat 

Agree 


Agree 


Strongly 

Agree 


Paying close attention to punctuation marks is necessary to understanding 
story content. 


□ 


□ 


□ 


□ 


□ 


It is a sign of an ineffective reader when words and phrases are repeated. 


□ 


□ 


□ 


□ 


□ 


Being able to label words according to grammatical function (nouns, etc.) is 
useful in proficient reading. 


□ 


□ 


□ 


□ 


□ 


When coming to a word that’s unknown, the reader should be encouraged to 
guess the meaning and go on. 


□ 


□ 


□ 


□ 


□ 


Young readers need to be introduced to the root form of a word (run, long) 
before they are asked to read inflected forms (running, longest). 


□ 


□ 


□ 


□ 


□ 


It is not necessary for a child to know the letters of the alphabet in order to 
learn to read. 


□ 


□ 


□ 


□ 


□ 


Flashcard drills with sight words is an unnecessary form of practice in 
reading instruction. 


□ 


□ 


□ 


□ 


□ 


Ability to use accent pattern in multi-syllable words (pho’to graph, photo’gra 
phy, and pho to gra’ phic) should be developed as part of reading instruction. 


□ 


□ 


□ 


□ 


□ 


Controlling text through consistent spelling patterns (The fat cat ran back. 
The fat cat sat on a hat.) is a means by which children can best learn to read. 


□ 


□ 


□ 


□ 


□ 


Formal instruction in reading is necessary to insure the adequate 
development of all the skills used in reading. 


□ 


□ 


□ 


□ 


□ 


Phonic analysis is the most important form of analysis used when meeting 
new words. 


□ 


□ 


□ 


□ 


□ 


Children’s initial encounters with print should focus on meaning, not upon 
exact graphic representation. 


□ 


□ 


□ 


□ 


□ 


Word shapes (word configuration) should be taught in reading to aid in word 
recognition. 


□ 


□ 


□ 


□ 


□ 


It is important to teach skills in relation to other skills. 


□ 


□ 


□ 


□ 


□ 


If a child says “house” for the written word “home,” the response should be 
left uncorrected. 


□ 


□ 


□ 


□ 


□ 


It is not necessary to introduce new words before they appear in the reading 
text. 


□ 


□ 


□ 


□ 


□ 


Some problems in reading are caused by readers dropping the inflectional 
ending from words (e.g., jumps, jumped^. 


□ 


□ 


□ 


□ 


□ 


6. Please tell us about your teaching style and 
your school's reading program. 


Strongly 

Disagree 


Disagree 


Somewhat 

Agree 


Agree 


Strongly 

Agree 


I make a special effort to give my students work that is creative and 
imaginative. 


□ 


□ 


□ 


□ 


□ 


I display the work of the highest achieving students as an example. 


□ 


□ 


□ 


□ 


□ 


I stress to students that I want them to understand the work, not just 
memorize it. 


□ 


□ 


□ 


□ 


□ 


I give special privileges to students who do the best academically. 


□ 


□ 


□ 


□ 


□ 


I frequently tell my students that I want them to enjoy learning. 


□ 


□ 


□ 


□ 


□ 


I help students understand how their performance compares to others. 


□ 


□ 


□ 


□ 


□ 


I make a special effort to give my students work that has meaning in their 
everyday lives. 


□ 


□ 


□ 


□ 


□ 


I point out those children who do well academically, as a model for other 
students. 


□ 


□ 


□ 


□ 


□ 


I encourage students to compete with each other academically. 


□ 


□ 


□ 


□ 


□ 
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Computers are an essential tool in teaching reading. 


□ 


□ 


□ 


□ 


□ 


Phonics should be taught every year in grades K-3. 


□ 


□ 


□ 


□ 


□ 




Strongly 

Disagree 


Disagree 


Somewhat 

Agree 


Agree 


Strongly 

Agree 


Teaching reading in science and social studies improves learning in those 
areas. 


□ 


□ 


□ 


□ 


□ 


Teaching phonics is a main goal of Reading First. 


□ 


□ 


□ 


□ 


□ 


Teaching sight words is a main goal of Reading First. 


□ 


□ 


□ 


□ 


□ 


Increasing reading comprehension is a main goal of Reading First. 


□ 


□ 


□ 


□ 


□ 


Using the computer to reinforce reading is a main goal of Reading First. 


□ 


□ 


□ 


□ 


□ 


I spend more time teaching phonics than I did before Reading First. 


□ 


□ 


□ 


□ 


□ 


Good reading instruction requires grouping students by ability level. 


□ 


□ 


□ 


□ 


□ 


Under Reading First, I have less time to teach writing. 


□ 


□ 


□ 


□ 


□ 


Under Reading First, I have less time to teach math. 


□ 


□ 


□ 


□ 


□ 


Reading First isn’t too different from how I’ve always taught. 


□ 


□ 


□ 


□ 


□ 


Most K-3 teachers at my school have low expectations for student academic 
achievement. 


□ 


□ 


□ 


□ 


□ 


The principal at my school believes in and supports Reading First. 


□ 


□ 


□ 


□ 


□ 


Most teachers at my school believe in and support Reading First. 


□ 


□ 


□ 


□ 


□ 


Computers are necessary to teach grammar. 


□ 


□ 


□ 


□ 


□ 


Computers are necessary to teach spelling. 


□ 


□ 


□ 


□ 


□ 


Computers are necessary to teach writing. 


□ 


□ 


□ 


□ 


□ 


I give students homework assignments that reinforce reading strategies they 
learn in class. 


□ 


□ 


□ 


□ 


□ 


The BLT is a good tool for assessing my students’ reading ability. 


□ 


□ 


□ 


□ 


□ 


The ITBS is a good tool for assessing my students’ reading ability. 


□ 


□ 


□ 


□ 


□ 


7. Please tell us about how your reading classes are structured. 


0-20 

min 


21-40 

min 


41-60 

min 


61-80 

min 


80+ 

min 


On average, how many minutes each DAY do you teach phonics? 


□ 


□ 


□ 


□ 


□ 


On average, how many minutes each DAY do you teach sight words? 


□ 


□ 


□ 


□ 


□ 


On average, how many minutes each DAY do you combine reading and 
science? 


□ 


□ 


□ 


□ 


□ 


On average, how many minutes each DAY do you combine reading and 
social studies? 


□ 


□ 


□ 


□ 


□ 


On average, how many minutes each DAY do you provide reading 
instruction in a small group? 


□ 


□ 


□ 


□ 


□ 


On average, how many minutes each WEEK does each of your students 
spend using the computer to reinforce reading skills? 


□ 


□ 


□ 


□ 


□ 


On average, how many minutes each WEEK do you use the computer to 
manage reading instruction? 


□ 


□ 


□ 


□ 


□ 


8. Please indicate the frequency in which the following occur: 


Almost 
every day 


Once or 
twice a 
week 


Once or 
twice a 
month 


Once or 
twice a 
semester 


Less than 


once or 
twice a 
semester 


How often do you assess your students’ reading levels? 


□ 


□ 


□ 


□ 


□ 


How often do your flexible groups change? 


□ 


□ 


□ 


□ 


□ 
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Study Questions 

Reading First Discussion Groups 

April - May 2001 

Note: These are generic questions that can be compared across groups. Where “reading program” 
is used, the term “Reading First” will be used for the Reading First groups. 

1 . What is your philosophy of teaching reading to primary level students? 

2. Describe the reading program used in Grades K-3 in your school/district. 

Reading First 

Other (Refer to reading programs included on survey) 

a) Why did your school/district decide to implement the Reading First Program? Has 
the Reading First Program met your expectations? Why or why not? 

b) Why did your school/district decide not to implement the Reading First Program? 

3. How is the reading program implemented in your school/ district? Request the following 
information for each program used in the school/district: 

Instructional practices 
Teaching techniques 
Strategies 
Organization 

Schedule/Time Spent on Reading 
Materials 

a) How does reading instruction differ by grade level? 

b) Is reading instruction integrated with instruction in other subject areas? If so, in what way? 

c) How are computers used for reading instruction? 

d) How are parents involved in the reading program? 

e) Do you have an advisory committee for the reading program? What is its role? How does 
the advisory committee influence the reading program? 

f) What staff development has been provided to reading teachers in your school? How 
satisfied are teachers with these activities? Do you have additional training needs? Please 
specify. 

4. What is the greatest strength of your reading program? 

a) What are the most effective instructional practices used? 

b) What are the most effective instructional materials used? 

(What do you like about the Reading First Program curriculum and materials? What don’t you 
like?) 




Georgia Department of Education 
Linda C. Schrenko, State Superintendent of Schools 
November 200 1 • Page 42 of 42 



48 



5. How effective do you believe your current reading program is? Why? 

a) What indicators does the school/district use to assess the reading program? 

b) Student achievement (standardized tests norm-referenced/criterion-referenced; district- 
level; teacher; performance-based) 

c) Other assessment indicators (student attitude, behavior, motivation, teacher reactions, 
parent reactions) 

6. How do you think each of the following groups views your current reading program? 

• Students 

• Parents 

• Teachers 

• Administrators 

7. How has your reading program (implementation, operation, curriculum, materials, etc.) changed 
over the past two school years? Why were the changes made? 

8. Will the school/district continue using the same reading program during the next school year? 
Why or why not? (Is change planned for your reading program for the next school year (2001- 
2002)?) Please describe the planned changes. Why are these changes being made? (Refer to 
reading programs included on survey) 

9. How well does the current reading program meet student needs? What student needs are not 
being met by the current reading program? 

10. How would you change/improve your reading program? Please cite specific recommendations for 

improvements needed. 

11. What other concerns do you have about Reading First or your reading program that we have not 

covered today in our discussion? 
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